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1. INTRODUCTION 

Research of the central nervous systems has attracted much attention recently, and a large number of 
contributions, especially theoretical ones, have been made by physicists since the 1980'sjj], 0, E|. In these 
researches, theoreticians have made very simplified models of the brain, and studied their statistical 
properties analytically as well as numerically. Among those models, the Hopfield model and its exten- 
sions have been throughly studied, and many interesting properties have been obtained, including the 
famous p/N ~ 0.14 limit where N is the number of neurons and p is the number of stored patterns [|[ 0]. 
An important hypothesis generally used on these models is the Hebb rule, that is to say, "the changes 
of synaptic strength are proportional to the correlation between the firing of the pre- and post-synaptic 
neurons" [El. The original idea by Hebb has been changed a little, i.e., firing of the post-synaptic neuron 
is not always necessary and the change of the membrane potential is more significant for synaptic plas- 
ticity . However there still remain many problems in the molecular process of synaptic transmission. 
The molecular basis of the Hebb rule has been pursued precisely to explain synaptic activity |7j] . 

Many qualitative behaviors of neural networks have been elucidated based on the Hebb rule mentioned 
above, but there still remain many differences between the real functions of the brain and artificial neural 
networks. The most important of them is that the real brain always works in the non-equilibrium states 
accompanied by nerve impulses. On the other hand, models of artificial neural networks have been 
treated in the equilibrium state of the thermo-dynamical limit. Thus, it is inevitable that the model of 
neural networks is extended to the non-equilibrium state. In this paper, we will propose a new model 
of neuro-flow dynamics in which the essential variable to describe the neural activity is the flow rate of 
neuro-impulses which propagate from the jth to the ith neuron. Furthermore, the original Hebb rule 
should be extended to include synaptic plasticity due to the neural flow. In this paper we will discuss 
some remarkable effects generated by the neuro-flow dynamics with the extended Hebb rule. 

2. MODEL OF NEURO-FLOW DYNAMICS 

The standard representation of neural networks in discrete time is written by 

N 

<Ti(t + 1) = w<i(*)°J-(*) - h i) (!) 

i=i 

where <7j and hi are the neuronal activity and the threshold of the «th neuron respectively, Wij is the 
element of the connection matrix which stands for the coupling strength from the jth to the ith neuron 
and /t(0 is a monotonically increasing function (0 < ir(') < 1) with a parameter T. Multiplying 
Wki{t + 1) to both sides of equation (1) and introducing a new variable Yki(t) — Wki(t)ai(t), equation 
(1) can be rewritten as 

N 

Y hl (t + 1) = w H (t + l)f T Q2 Yij{t) - hi). (2) 

j=i 

Here Y^ describes the flow rate of impulse transmission from the jth to the ith neuron. The variable Yij 
represents the strength of the impulse flow qualitatively and equation (2) describes the essential effect 
of the neuro-flow, but the dynamics of the flow variable Yij given by equation (2) cannot be uniquely 
determined. In order to recover the deterministic equation of the flow variable Y^, let us replace the 



term w k i(t + 1) by Wki{t) in equation (2). Thus, the Markovian equation of the flow variable instead 
of Yij is obtained 

N 

X ki (t+1) = w kl (t).f T (J2 X v(t) - hi)- (3) 

Since our purpose is to elucidate the effect of the neuro-ffow, both equation (2) and equation (3) should 
be the starting point for that purpose. But from the view point of the consistent description of the 
system, we consider that equation (3) is more natural than equation (2). For the sake of simplicity, the 
dynamics described by equation (1) is called the pattern dynamics, on the other hand, the dynamics 
given by equation (3) is known as the neuro-flow dynamics in this paper. We rewrite equation (3) using 
Xki{t) = Wki(t — l)<7j(i), the pattern dynamics of the variable Oi corresponding to equation (3) can be 
given by 

JV 

0i{t + 1) = /rE%(i - l)aj(t) - hi). (4) 

This explains the essential difference between equations (1) and (4), that is to say, equation (1) is sim- 
ple Markovian, but equation (4) is not. Thus, the flow dynamics of equation (3) can be expected to 
reveal quite complex aspects in comparison with the pattern dynamics of equation (1). To summarize, 
the neuro-flow dynamics, introduced by equation (3), is equivalent to the pattern dynamics with time- 
delayed effects. 



3. EXTENSION OF THE HEBB RULE 

The Hcbb rule is often written by 

Wij (t + 1) = A Wij (t) + C(2Cf - 1)(2C; - 1) (5) 

where A is the decreasing rate C is the learning acquisition rate, and the second term of the right 
hand side stands for the learning term generated by the learning patterns {Cf } (Cf = 1 or with equal 
probability and fj, = 1, ...,p). To store the learning patterns as local minima, the learning term 2C^ — f 
is used instead of Here we rewrite £f = 2Q? — 1 and extend equation (5) by taking account the flow 
effect as follows 

wn (t + 1) = Awij (t) + BXij (t + 1) + C#$. (6) 

In this paper, we only discuss the case of B > according to the original Hebb's idea, equation (6) is 
the simplest extension which introduces the flow effect of the nerve transmission. 

Dynamical systems given by equations (3) and (6) can be solved by iteration. Let us show the solution 
for the special case hi = 0, where the initial condition is given by t%(0) = 0, Xij(0) = and fr{ ) is a 
monotonically increasing function satisfying /t(0) = 1/2, /t(+oo) = 1 and /t(— oo) = 0. The first two 
steps in the time course can be obtained 

TV 

X ki (l) = w k i(0)f T (J2 X ioi°)) 
= 

w ki (l) = Aw k i(0)+BX ki (l)+C^l 

= (7) 

N 

X kl (2) = Wki^frC^Xi^l)) 
w kl (2) = Aw kl {l) + BX kt {2) + Ce k g 

= c{(A+^)e k g+e k g}. (8) 

Here we consider two cases; One is the special case that the value £f of each pattern takes ±1 randomly 
with the constraint J2j=i €j ~^ as N — > oo (Case 1). The other is the ordinary case where the value 



£f takes ±1 randomly, i.e. Y,f=i ^ ~ 0(\/iV) as JV -» oo (Case 2). 
3.1. Case 1 

If we assume the value of each pattern takes ±1 under the constraint that the pattern fluctuations 
are negligibly small in the limit of large system size N (i.e. Y^j=i £^ ^ as A r ^ oo), the solutions of 
equations (3) and (6) for the third time step can be analytically obtained 

N 

X kl (3) = Wfel (2)/ T (]Tx y (2)) 

r r N 

w kl (3) = Aw kl (2) + BX kl (3) + C^f 

= c{(A+^fe k zi + (A+^)e k tf+m- m 



After the iteration of p patterns, the solutions are 

x k M = f E^ + f) (p_1) " M « 

Wkl {p) = cj^iA + ^y-^- (io) 

For the case of the normal Hebb rule (i.e. B = 0) the last few learning patterns }, /it ~ p) can be well 
stored due to the condition A < 1. But when the flow terms are introduced, i.e. (A + B/2) > 1, the first 
few learning patterns ~ 1) determine the connection matrix Wij dominantly. As a result, the 

other learning patterns cannot be stably stored. We will explain this point later by numerical simulation. 

3.2. Case 2 

On the other hand, in the case of the more general condition where we assume that the value £f of 
each pattern takes ±1 randomly, the solutions of equations (3) and (6) for the third time step change as 
follows: 

N 

A fel (3) = w ki (2)fT(J2 X ij( 2 )) 

= c{(a + + zimfH^i x ±o(Vn)) 

w kl (3) = Aw kl (2) + BX kl (3) + CXU! 

= c{i(A+j)e k £+e k &}{A+j(i± ti)]+e k m- (ii) 

After the iteration of p patterns, the solutions are 

Xki (p) = C{(A+^l(aty- s /^(af^-^n^ 

fj,=2 

w ki {p) = c{(A+|)^^(af i r 2 +^(a± i r^n (12) 

fj,=2 

where af 1 = A + B{1 ± (l)/2. Thus, in the case of the random learning patterns without constraints, 
the connection matrix is determined by af 1 instead of (A + B/2). For the case of the normal Hebb rule 



o 



(B = 0), the last few learning patterns can be well stored since A < 1, and the matrix element Wij(p) of 
equation (12) becomes the same as that of equation (10). However, when B > 0, af 1 takes (A + B) or A. 
This implies that the last few learning patterns can be well stored for (A + B) < 1, but for (A + B) > 1 
the first few learning patterns can be stored. We will explain this point later by numerical simulation. 

4. SLEEP IN NEURO-FLOW DYNAMICS 

Now let us further investigate equation (6) under the condition of no learning term, i.e. C = 0. When 
the external stimuli stop, the neural network evolves autonomously according to the neuro-flow dynamics 
equations (3) and (6) with C = 0. This situation can be compared with the state where the central ner- 
vous systems are sleeping, that is, the activity of brain is self-organized only by the internal information. 
The time evolution of Wij during sleep can be written as 

w ki (t+l) = Aw ki (t) + BX kl (t + 1) 

= Wki^A + BfTC^X^-hi)]. (13) 

In the case of the ordinary Hebb rule (B = 0), Wij(t) vanishes when t — > oo because of A < 1, but in 
the case of B > 0, Wij(t) does not always decrease. In other words, sleep in the neuro-flow dynamics 
brings about two quite different effects; one is forgetting of the memory, the other is reinforcement of 
the memory. The general criterion for forgetting or reinforcement is derived from equation (13): 

1-A 



w k t(t+ 1) > w k i(t); for f T (^2Xij(t) - hi) > 



B 
' A 



w k i(t + 1) < w ki (t): for f T (^rXij(t) - hi) < - 



(14) 



B 



We discuss the time evolution of X ki (t) and w k i(t) in the sleep condition for Case 1 and Case 2 as 
mentioned before. 

4.1. Case 1 

In the case of the random patterns with the constraint Y^=i £j — * as N — * oo, after the learning 
obtained by equation (10) is finished, the addition of the I times sleep induces the following solutions: 

CA,, B, 



w kl (p + l) = C^2(A+^r+ 1 -^. (15) 

Here in the case of (A + B/2) > 1, the matrix elements My corresponding to the first few learning 
patterns can be much amplified after sleep. This is nothing but the reinforcement of the primacy effect 
[8] by sleep. On the other hand, every learning pattern is vanishing for (A + B/2) < 1. 

4.2. Case 2 

In the case of random patterns without constraints, the addition of the / times sleep after the learning 
of equation (12) induces the following solutions 

w kl (p + l) = C{(A+^)e k ^(af^ +l - 2 + j2(arr +l -^}. (16) 

Here every learning pattern will be erased for af 1 < 1, but the first few learning patterns can be ampli- 
fied for af 1 > 1; the condition of the reinforcement of the primacy effect will be given by (A + B) > 1. 



5. NUMERICAL SIMULATION 



Let us now numerically investigate the neuro-flow dynamics with the extended Hebb rule. The values 
of the parameters used in this paper are fixed as follows: N — 100, p — 10, hi — 0; the orthogonal 
learning patterns {£f }s are selected from the randomly produced patterns and they satisfy YljLi £j ~ 

and J2f=i S,j is set to for Case 1. f T (x) is fixed as f T (x) = 1/(1 + exp(-x/T)) with T = 0.05. The 
time development is performed by the synchronous updating of equations (3) and (6) under the initial 
conditions of Wy (0) = and -Xy(O) = 0. 

5.1. Case 1 

The connection matrix Wij(p) were determined after the learning process mentioned above. The basin 
structure corresponding to the fixed matrix Wij(p) was searched by the Monte-Carlo simulations, i.e. 
initial values of the test flow Xy(Q)s in equation (3) were distributed randomly in the range — 1 < 
X%j(0) < 1. The statistical properties of the basin structure can be represented by the rank-size relation 
of each pattern; the size is defined by the volume of the each basin and the rank of each basin is defined 
in accordance with the order of largeness of the basin volume. For example, rank 1 corresponds to the 
largest basin and rank 2 to the second largest basin, etc. Figure 1 shows the relation between rank n 
and the basin volume V for the case of A = 1 and B = 0. Here the filled circles stand for the learning 
patterns and the crosses for the spurious states. As is shown in Fig. 1, there exist a number of basins 
for spurious states and they obey 

V oc n-° (D = 0.5582152) (17) 

where D is the Zipf index. However all learning patterns have a larger basin volume than that of spurious 
states, and this clearly shows that the neuro-flow dynamics proposed here has the learning ability in 
the same level as the Hopfield model of neural networks. Furthermore, the rank-size relation obeys an 
inverse power law. This is also similar to the result that was reported in a previous paper treating the 
Hopfield model §, [IcJ. 

When the flow term is introduced, the basin structure is drastically changed as is shown in Figs 2 and 
3. Figure 2 reveals the flow effect on the learning process in the (A, B) parameter space. The number 
of stably stored learning patterns M(M < p) is shown in the two-dimensional (A, B) parameter space 
in gray-scale. The axis of B = corresponds to the case of the normal Hebb rule. All learning patterns 
are stably stored around (A + B/2) = 1 and the number of stably stored learning patterns M decreases 
almost monotonically as the distance from the line (^4 + B/2) — 1 increases. Figure 3 shows the change 
of the basin volume of each learning pattern (/i = 1, 2, 10) for the various values of B at A = 0.9. 
We can see that when the value of B increases, the basin volume for the first few learning patterns 
(/i = 1, 2, 3) become dominant for B > 0.2 and the basin for spurious states gets smaller (to less than 
10%). This result clearly reveals the primacy effect of the memory and is consistent with the theoretical 
prediction derived from equation (10). On the other hand, for the case of B < 0.2, the basin volume 
for the last few learning patterns (/i = 8, 9, 10) become dominant. In other words, a transition occurs 
from the recency effect (§) to the primacy effect at the critical point B ~ 0.2. 

Next, further iteration after the learning (p = 10) by the neuro-flow dynamics without the learning 
term was performed numerically using equations (3) and (6) with I = 1000. Figures 4 and 5 show the 
results of the basin structures and the phase diagram. Comparing Fig. 4 with Fig. 2, one can see that 
memories are completely erased after sleep in the region of (A + B/2) < 1. Figure 5 shows the change 
of the basin volume for each learning pattern, where the value of parameter A is the same as in Fig. 3. 
The basin volume for the first few learning patterns (fi — 1, 2, 3) increases much more sharply and the 
spurious basin becomes smaller very quickly for B > 0.2. On the other hand, in the case of B < 0.2, 
the basin volume for all learning patterns is vanishing. This shows that the primacy effect discussed in 
Fig. 3 is much more emphasized due to the sleep. Figure 5 appears to suggest that sleep reinforces and 
selects memories which were learned during awakeness, and that a lot of spurious memories are washed 
out during sleep. Actually the system approaches steady states after long sleep when the effects of the 
flow increases and the number of spurious memories becomes less than 10%. 

5.2. Case 2 

For the case of random patterns without constraints, the same numerical simulation was performed and 
quite different property was elucidated. 

Figure 6 shows the relation between rank n and the basin volume V for the case of A = 1 and B = 0, 



and spurious states obey 

Vocn~ D (D = 0.5582152). (18) 

Here the learning ability of neuro-fiow dynamics is clearly shown again by the fact that all the learning 
patterns have a larger basin volume than that of spurious states. As was expected from the theoretical 
analysis, the slope of Fig. 1 and that of Fig. 6 are completely same. 

The flow effect on the learning process is shown in Fig. 7 and it is quite different from Fig. 2. The 
line (A + B) = 1 clearly separates two region: (A + B) < 1 and (A + B) > 1. In {A + E) < 1, all 
the learning patterns are stably stored at A w 1, in which (A + B) « A, and gradually decrease as A 
decreases. However, the number of stably stored patterns shows a complex structure depending on the 
combination of the value of (A + B) and A. On the other hand, in (A + B) > 1 only one pattern is stored 
successfully, and this can be understood from equation (12). Since af 1 takes (^4 + B) or A according to 
the value and since (A + B) > 1 and A < 1, all the terms (/i = 2, 3, • • • , p) is totally disturbed 
and cannot be stably stored. 

Figure 8 shows the change of the basin volume of each learning pattern for the various values of B 
at A — 0.9. We can clearly see the drastic change from the recency effect for B < 0.1 to the primacy 
effect for B > 0.1 occurs. It is also surprising that all initial flows converge to the first learning pattern 
(/i = 1) for B > 0.1 and all spurious states are washed out. 

Next, the further iteration after learning (p = 10) was performed in the sleep condition with / = 1000. 
Figures 9 and 10 show the results of the basin structure and the phase diagram. As was seen in Fig. 
4, the erasure of memories is also observed for (A + B) < 1 in Fig. 9. Figure 10 shows the change of 
the basin volume for each learning pattern, where the value of parameter A is the same as in Fig. 8. 
We can see that a much sharper change than Fig. 8 occurs at B = 0.1, that is, for B < 0.1 all learning 
patterns are erased, and for B > 0.1 the first learning pattern (/i = 1) becomes dominant. It should be 
also pointed out that there are no spurious states in B > 0.1. 

The above results indicate that sleep amplifies the primacy effect in the case of random patterns. 
Therefore, together with the results of Case 1, it can be suggested that sleep reinforces and selects the 
memories which were learned during awakeness, and that many spurious memories are erased during the 
sleep. 



6. SUMMARY 

We have proposed a neuro-fiow dynamics with the extended Hebb rule to deal with the autonomy of 
the neural systems and have shown that the neuro-fiow dynamics is equivalent to the pattern dynamics 
with a certain time delayed connection. Two essential new features are derived from the neuro-fiow 
dynamics. One is the reinforcement of the primacy effect of the memory, the other is the striking effect 
of sleep which washes out the spurious memories. The latter point is quite similar to the idea that was 
mentioned by Click concerning the function of dreaming in sleep fllf . In this connection, the mechanism 
of the memory palimpsest, which was studied by Nadal et al. |lSj| , in the future must be discussed in 
comparison with our neuro-fiow dynamics model. 

The analysis in this paper is only limited to the case only for random learning patterns and their spe- 
cial case. The extension to the non-random patterns will be discussed in a forthcoming paper together 
with the generalization of the neuro-fiow dynamics model |]l3| . 
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Fig. 1. The rank-size relation of Case 1 for the basin volume obtained by Monte-Carlo simulation with 
10000 randomly chosen initial conditions. The parameters are A = 1.0, B = 0.0 and C = 1.0. The filled 
circles correspond to the learning patterns and the crosses to spurious states. The solid line shows the 
fit of the spurious states by the least-squares method and the slope is —0.5582152. 

Fig. 2. Formation of the basin for the 10 random learning patterns with the constraint Y^j=i €j = 0- 
A and B are varied between and 1 with 0.01 steps. The gray-scaled M(M < 10) stands for the num- 
ber of stably stored learning patterns. In the region of (A + B/2) > 1, the first few learning patterns 
(/i = 1, 2, 3) are stable, but in the region of (A + B/2) < 1, the last few learning patterns (fi = 8, 9, 10) 
are stable. The result is successfully explained by equation (10). 

Fig. 3. The basin volume for each learning pattern (fi = 1, 2, 10) obtained by Monte-Carlo simulation 
with 1000 randomly chosen initial conditions. The parameters are A = 0.9 and C = 1.0. The index \i is 
written at the curve which corresponds to the recency or the primacy effect. 

Fig. 4. Formation of the basin for the 10 random learning patterns with the constraint X^jLi £j* = a fter 
sleep; p = 10 and I = 1000. A and B are varied between and 1 with 0.01 steps. All learning patterns are 
only stably stored around the line (A + B/2) = 1 which is consistent with equation (13). In the region of 
(A + B/2) < 1, memories for all learning patterns were completely washed out by sleep and in the region 
of (A + B/2) > 1, the first few learning patterns (fi — 1, 2, 3) remain very strongly even after a long sleep. 

Fig. 5. The basin volume for each learning pattern after sleep; p — 10 and I = 1000. The basin volume 
for the first few learning patterns increases very sharply when the value of B increases. This implies 
reinforcement of the primacy effect by sleep. The parameters are A = 0.9 and C = 1.0. The index /j, is 
written at the curve which corresponds to the primacy effect. 

Fig. 6. The rank-size relation of Case 2 for the basin volume obtained by Monte-Carlo simulation with 
10000 randomly chosen initial conditions. The parameters are A = 1.0, B = 0.0 and C = 1.0. The filled 
circles correspond to the learning patterns and the crosses to spurious states. The solid line is the fit of 
the spurious states by the least-squares method and the slope is —0.5582152. This figure is the same as 
Figure 1 because the matrix element Wij(p) of equations (10) and (12) becomes equal for B = 0. 

Fig. 7. Formation of the basin for the 10 random learning patterns. A and B are varied between and 
1 with 0.01 steps. In the region of (A + B) > 1, only the first learning pattern (/i = 1) is stable, but in 
the region of (A + B) < 1, all learning patterns are stably stored at A w 1 and decrease as A decreases. 
The result is explained qualitatively well by equation (12). 

Fig. 8. The basin volume for each learning pattern (/i = 1, 2, 10) obtained by Monte-Carlo simulation 
with 1000 randomly chosen initial conditions. The parameters are A = 0.9 and C = 1.0. The index \i is 
written at the curve which corresponds to the recency or the primacy effect. 

Fig. 9. Formation of the basin for the 10 random learning patterns after sleep; p = 10 and I — 1000. A 
and B are varied between and 1 with 0.01 steps. In the region of (A + B) < 1, memories for all learning 
patterns were completely washed out by sleep and in the region of (A + B) > 1, the first learning pattern 
(fi = 1) remains very strong even after a long sleep. 

Fig. 10. The basin volume for the 10 learning patterns after sleep; p — 10 and I — 1000. The basin 
volume for the first learning pattern increases very sharply when the value of B increases. This implies 
reinforcement of the primacy effect by sleep. The parameters are A = 0.9 and C = 1.0. The index [i is 
written at the curve which corresponds to the primacy effect. 
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